Adding support of TIME64 datatype #3697

jgao54 · 2025-11-15T00:44:56Z

Postgres TIME supports range: [00:00:00, 24:00:00], with microsecond precision
MySQL TIME supports range: [-838:59:59.000000, 838:59:59.999999], with microsecond precision
ClickHouse Time64 supports range: [-999:59:59.000000, 999:59:59.999999], with nanosecond precision

This implementation for Time64 allow ClickHouse to fully support Postgres and MySQL Time ranges.

One subtle, but IMO useful improvement from this PR (aside from adding Time64 support) that's worth calling out:
we used to have Postgres convert to QValueTime while MySQL convert to QValueTimestamp prior to Time64 was introduced; meaning Posgres handles conversion to DateTime64 in the destination connector, while MySQL handles the conversion in the source connector. With this PR, conversion is consistently handled in the destination connector. So we always convert Time -> QValueTime, and then handle backwards-compatibility by optionally converting QValueTime -> DateTime64 if on an older version of peerdb or if on an older version of ClickHouse that does not support Time64.

I've also evaluated bringing consistency by aligning Postgres to MySQL, and do the branching when mapping Time (source type) -> QValue, and keep the destination consistent instead. This is a messier approach, since we can't convert to Time64 unless it is supported by ClickHouse, the source connector needs to know about ClickHouse version.

codecov · 2025-11-15T00:59:19Z

❌ 5 Tests Failed:

Tests completed	Failed	Passed	Skipped
1648	5	1643	163

View the top 3 failed test(s) by shortest run time

github.com/PeerDB-io/peerdb/flow/e2e::TestApiMy

Stack Traces | 0.01s run time

=== RUN   TestApiMy
=== PAUSE TestApiMy
=== CONT  TestApiMy
--- FAIL: TestApiMy (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg

Stack Traces | 0.01s run time

=== RUN   TestApiPg
=== PAUSE TestApiPg
=== CONT  TestApiPg
--- FAIL: TestApiPg (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteBQ

Stack Traces | 0.01s run time

=== RUN   TestPeerFlowE2ETestSuiteBQ
=== PAUSE TestPeerFlowE2ETestSuiteBQ
=== CONT  TestPeerFlowE2ETestSuiteBQ
--- FAIL: TestPeerFlowE2ETestSuiteBQ (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH_Cluster

Stack Traces | 0.01s run time

=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH_Cluster
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH_Cluster
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH_Cluster
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH_Cluster (0.01s)
2026/01/22 09:37:16 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:37:16 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:37:16 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgchcl_s8s0lecb'"
2026/01/22 09:37:16 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgchcl_s8s0lecb'"
2026/01/22 09:37:16 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_2993357031767995650 CURSOR FOR SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgchcl_s8s0lecb'" args=[]
2026/01/22 09:37:16 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgchcl_s8s0lecb'" channelLen=0
2026/01/22 09:37:16 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2993357031767995650
2026/01/22 09:37:16 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2993357031767995650 records=0 bytes=0 channelLen=0
2026/01/22 09:37:16 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgchcl_s8s0lecb'" rows=0 bytes=0 channelLen=0
2026/01/22 09:37:16 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:37:16 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT sync_batch_id FROM metadata_last_sync_state WHERE job_name='test_normalize_metadata_with_retry_pgchcl_s8s0lecb'" rows=0 bytes=0 channelLen=0
2026/01/22 09:37:16 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgchcl_00q2ysxk.\"test_infinite_time\" ORDER BY id"
2026/01/22 09:37:16 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgchcl_00q2ysxk.\"test_infinite_time\" ORDER BY id"
2026/01/22 09:37:16 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_5619778489849425280 CURSOR FOR SELECT id FROM e2e_test_pgchcl_00q2ysxk.\"test_infinite_time\" ORDER BY id" args=[]
2026/01/22 09:37:16 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgchcl_00q2ysxk.\"test_infinite_time\" ORDER BY id" channelLen=0
2026/01/22 09:37:16 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5619778489849425280
2026/01/22 09:37:16 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5619778489849425280 records=2 bytes=8 channelLen=1
2026/01/22 09:37:16 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgchcl_00q2ysxk.\"test_infinite_time\" ORDER BY id" rows=2 bytes=8 channelLen=0
2026/01/22 09:37:16 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5619778489849425280
2026/01/22 09:37:16 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5619778489849425280 records=0 bytes=0 channelLen=0
2026/01/22 09:37:16 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgchcl_00q2ysxk.\"test_infinite_time\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/01/22 09:37:16 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:37:16 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id FROM e2e_test_pgchcl_00q2ysxk.\"test_infinite_time\" ORDER BY id" rows=2 bytes=8 channelLen=0

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuitePG_CH_Cluster

Stack Traces | 0.01s run time

=== RUN   TestPeerFlowE2ETestSuitePG_CH_Cluster
=== PAUSE TestPeerFlowE2ETestSuitePG_CH_Cluster
=== CONT  TestPeerFlowE2ETestSuitePG_CH_Cluster
--- FAIL: TestPeerFlowE2ETestSuitePG_CH_Cluster (0.01s)
2026/01/22 09:38:42 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:38:42 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:38:43 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,ky,val,n,t FROM e2e_test_pgch_ziznd5ei.\"test_nullable_mirror\" ORDER BY id"
2026/01/22 09:38:43 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,ky,val,n,t FROM e2e_test_pgch_ziznd5ei.\"test_nullable_mirror\" ORDER BY id"
2026/01/22 09:38:43 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_10519337171534228144 CURSOR FOR SELECT id,ky,val,n,t FROM e2e_test_pgch_ziznd5ei.\"test_nullable_mirror\" ORDER BY id" args=[]
2026/01/22 09:38:43 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,ky,val,n,t FROM e2e_test_pgch_ziznd5ei.\"test_nullable_mirror\" ORDER BY id" channelLen=0
2026/01/22 09:38:43 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10519337171534228144
2026/01/22 09:38:43 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10519337171534228144 records=2 bytes=15 channelLen=1
2026/01/22 09:38:43 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,ky,val,n,t FROM e2e_test_pgch_ziznd5ei.\"test_nullable_mirror\" ORDER BY id" rows=2 bytes=15 channelLen=1
2026/01/22 09:38:43 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10519337171534228144
2026/01/22 09:38:43 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10519337171534228144 records=0 bytes=0 channelLen=0
2026/01/22 09:38:43 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,ky,val,n,t FROM e2e_test_pgch_ziznd5ei.\"test_nullable_mirror\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/01/22 09:38:43 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:38:43 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,ky,val,n,t FROM e2e_test_pgch_ziznd5ei.\"test_nullable_mirror\" ORDER BY id" rows=2 bytes=15 channelLen=0

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_MySQL_Time

Stack Traces | 31.1s run time

=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_MySQL_Time
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_MySQL_Time
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_MySQL_Time
2026/01/22 09:36:20 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:36:21 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:36:21 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_api_veev9kcm.t1
    clickhouse_mysql_test.go:106: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_MySQL_Time (31.14s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_Time64

Stack Traces | 31.2s run time

=== RUN   TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_Time64
=== PAUSE TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_Time64
=== CONT  TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_Time64
2026/01/22 09:36:01 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:36:01 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:36:02 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_mychcl_jn6pwllj.test_nullable_mirror
    clickhouse_test.go:1339: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
--- FAIL: TestPeerFlowE2ETestSuiteMySQL_CH_Cluster/Test_Time64 (31.18s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Time64

Stack Traces | 31.2s run time

=== RUN   TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Time64
=== PAUSE TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Time64
=== CONT  TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Time64
2026/01/22 09:37:49 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:37:49 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    clickhouse_test.go:1339: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    clickhouse.go:114: begin tearing down postgres schema pgchcl_hq6grz3u
--- FAIL: TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Time64 (31.18s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Types_CH

Stack Traces | 31.2s run time

=== RUN   TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Types_CH
=== PAUSE TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Types_CH
=== CONT  TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Types_CH
2026/01/22 09:37:39 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:37:39 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:37:39 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_e7jcwtc9.\"test_update_pkey_chunking_initial_load_enabled\" ORDER BY id"
2026/01/22 09:37:39 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_e7jcwtc9.\"test_update_pkey_chunking_initial_load_enabled\" ORDER BY id"
2026/01/22 09:37:39 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_6931762045321300769 CURSOR FOR SELECT id,\"key\" FROM e2e_test_pgchcl_e7jcwtc9.\"test_update_pkey_chunking_initial_load_enabled\" ORDER BY id" args=[]
2026/01/22 09:37:39 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_e7jcwtc9.\"test_update_pkey_chunking_initial_load_enabled\" ORDER BY id" channelLen=0
2026/01/22 09:37:39 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6931762045321300769
2026/01/22 09:37:39 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6931762045321300769 records=4 bytes=40 channelLen=3
2026/01/22 09:37:39 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_e7jcwtc9.\"test_update_pkey_chunking_initial_load_enabled\" ORDER BY id" rows=4 bytes=40 channelLen=0
2026/01/22 09:37:39 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6931762045321300769
2026/01/22 09:37:39 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_6931762045321300769 records=0 bytes=0 channelLen=0
2026/01/22 09:37:39 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_e7jcwtc9.\"test_update_pkey_chunking_initial_load_enabled\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/01/22 09:37:39 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:37:39 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_e7jcwtc9.\"test_update_pkey_chunking_initial_load_enabled\" ORDER BY id" rows=4 bytes=40 channelLen=0
    clickhouse_test.go:1139: UNEXPECTED STATUS TIMEOUT STATUS_SETUP
    clickhouse.go:114: begin tearing down postgres schema pgchcl_ttxzswm1
2026/01/22 09:38:10 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_tiungs20.\"test_schema_as_column\" ORDER BY id"
2026/01/22 09:38:10 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_tiungs20.\"test_schema_as_column\" ORDER BY id"
2026/01/22 09:38:10 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_16567107544313255141 CURSOR FOR SELECT id,\"key\" FROM e2e_test_pgchcl_tiungs20.\"test_schema_as_column\" ORDER BY id" args=[]
2026/01/22 09:38:10 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,\"key\" FROM e2e_test_pgchcl_tiungs20.\"test_schema_as_column\" ORDER BY id" channelLen=0
--- FAIL: TestPeerFlowE2ETestSuitePG_CH_Cluster/Test_Types_CH (31.18s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiMy/TestQRep

Stack Traces | 63.2s run time

=== RUN   TestApiMy/TestQRep
=== PAUSE TestApiMy/TestQRep
=== CONT  TestApiMy/TestQRep
2026/01/22 09:34:37 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:34:37 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:1544: WaitFor qrep initial load 2026-01-22 09:34:37.228578164 +0000 UTC m=+272.952081297
2026/01/22 09:34:37 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_api_6mtvaw3n.qrepapi_api_6mtvaw3n
    api_test.go:1544: code: 60, message: Unknown table expression identifier 'qrepapi_api_6mtvaw3n' in scope SELECT id, val FROM qrepapi_api_6mtvaw3n FINAL WHERE _peerdb_is_deleted = 0 ORDER BY 1 ASC SETTINGS use_query_cache = false
    api_test.go:1549: WaitFor insert post qrep initial load 2026-01-22 09:34:38.238976553 +0000 UTC m=+273.962479686
2026/01/22 09:34:38 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_api_6mtvaw3n.qrepapi_api_6mtvaw3n
    api_test.go:1549: q.NumRecords: 2
    api_test.go:1549: other.NumRecords: 1
    api_test.go:1549: q.NumRecords: 2
    api_test.go:1549: other.NumRecords: 1
2026/01/22 09:34:39 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_api_kt58shot.valid
    api_test.go:1561: WaitFor finish 2026-01-22 09:34:40.258752767 +0000 UTC m=+275.982255910
2026/01/22 09:34:40 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_api_kt58shot.valid
    api_test.go:1561: UNEXPECTED TIMEOUT finish 2026-01-22 09:35:40.373989939 +0000 UTC m=+336.097493102
    api_test.go:44: begin tearing down postgres schema api_6mtvaw3n
--- FAIL: TestApiMy/TestQRep (63.20s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteBQ/Test_Complete_QRep_Flow_Avro

Stack Traces | 183s run time

=== RUN   TestPeerFlowE2ETestSuiteBQ/Test_Complete_QRep_Flow_Avro
=== PAUSE TestPeerFlowE2ETestSuiteBQ/Test_Complete_QRep_Flow_Avro
=== CONT  TestPeerFlowE2ETestSuiteBQ/Test_Complete_QRep_Flow_Avro
2026/01/22 09:33:43 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_pwxpdhfu.\"test_partition_noroot\" ORDER BY id"
2026/01/22 09:33:43 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_pwxpdhfu.\"test_partition_noroot\" ORDER BY id"
2026/01/22 09:33:43 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_4565137307862900658 CURSOR FOR SELECT id,name,created_at FROM e2e_test_pgchg_pwxpdhfu.\"test_partition_noroot\" ORDER BY id" args=[]
2026/01/22 09:33:43 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_pwxpdhfu.\"test_partition_noroot\" ORDER BY id" channelLen=0
2026/01/22 09:33:43 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4565137307862900658
2026/01/22 09:33:43 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4565137307862900658 records=13 bytes=293 channelLen=12
2026/01/22 09:33:43 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_pwxpdhfu.\"test_partition_noroot\" ORDER BY id" rows=13 bytes=293 channelLen=0
2026/01/22 09:33:43 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4565137307862900658
2026/01/22 09:33:43 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4565137307862900658 records=0 bytes=0 channelLen=0
2026/01/22 09:33:43 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_pwxpdhfu.\"test_partition_noroot\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/01/22 09:33:43 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:33:43 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_pwxpdhfu.\"test_partition_noroot\" ORDER BY id" rows=13 bytes=293 channelLen=0
    bigquery_qrep_test.go:73: WaitFor finish 2026-01-22 09:33:44.823113447 +0000 UTC m=+220.546616590
2026/01/22 09:33:44 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_xm5xug0y.\"test_inheritance_dynconf\" ORDER BY id"
2026/01/22 09:33:44 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_xm5xug0y.\"test_inheritance_dynconf\" ORDER BY id"
2026/01/22 09:33:44 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_4366418698922866302 CURSOR FOR SELECT id,name,created_at FROM e2e_test_pgchg_xm5xug0y.\"test_inheritance_dynconf\" ORDER BY id" args=[]
2026/01/22 09:33:44 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_xm5xug0y.\"test_inheritance_dynconf\" ORDER BY id" channelLen=0
2026/01/22 09:33:44 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4366418698922866302
2026/01/22 09:33:44 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4366418698922866302 records=8 bytes=168 channelLen=7
2026/01/22 09:33:44 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_xm5xug0y.\"test_inheritance_dynconf\" ORDER BY id" rows=8 bytes=168 channelLen=7
2026/01/22 09:33:44 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4366418698922866302
2026/01/22 09:33:44 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4366418698922866302 records=0 bytes=0 channelLen=0
2026/01/22 09:33:44 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_xm5xug0y.\"test_inheritance_dynconf\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/01/22 09:33:44 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:33:44 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_pgchg_xm5xug0y.\"test_inheritance_dynconf\" ORDER BY id" rows=8 bytes=168 channelLen=0
    bigquery_qrep_test.go:73: UNEXPECTED TIMEOUT finish 2026-01-22 09:36:45.189179432 +0000 UTC m=+400.912682565
    bigquery.go:86: begin tearing down postgres schema bq_tnjmsqqh_20260122093343
--- FAIL: TestPeerFlowE2ETestSuiteBQ/Test_Complete_QRep_Flow_Avro (182.56s)

github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuiteBQ/Test_PeerDB_Columns_QRep_BQ

Stack Traces | 183s run time

=== RUN   TestPeerFlowE2ETestSuiteBQ/Test_PeerDB_Columns_QRep_BQ
=== PAUSE TestPeerFlowE2ETestSuiteBQ/Test_PeerDB_Columns_QRep_BQ
=== CONT  TestPeerFlowE2ETestSuiteBQ/Test_PeerDB_Columns_QRep_BQ
    bigquery_qrep_test.go:142: WaitFor finish 2026-01-22 09:34:20.72213459 +0000 UTC m=+256.445637733
    bigquery_qrep_test.go:142: UNEXPECTED TIMEOUT finish 2026-01-22 09:37:21.090840393 +0000 UTC m=+436.814343536
    bigquery.go:86: begin tearing down postgres schema bq_ont0jb6l_20260122093419
--- FAIL: TestPeerFlowE2ETestSuiteBQ/Test_PeerDB_Columns_QRep_BQ (182.65s)

View the full list of 1 ❄️ flaky test(s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestResyncFailed

Flake rate in main: 19.05% (Passed 68 times, Failed 16 times)

Stack Traces | 138s run time

=== RUN   TestApiPg/TestResyncFailed
=== PAUSE TestApiPg/TestResyncFailed
=== CONT  TestApiPg/TestResyncFailed
2026/01/22 09:20:38 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/01/22 09:20:38 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:848: WaitFor wait for MV error messages 2026-01-22 09:20:38.367189226 +0000 UTC m=+484.025069515
2026/01/22 09:20:38 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('resync_failed_api_wjxjvrku' in flow_name) > 0\n\t\tAND error_message ILIKE '%while pushing to view e2e_test_api_wjxjvrku.resync_failed_api_wjxjvrku_mv_api_wjxjvrku%'"
2026/01/22 09:20:38 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('resync_failed_api_wjxjvrku' in flow_name) > 0\n\t\tAND error_message ILIKE '%while pushing to view e2e_test_api_wjxjvrku.resync_failed_api_wjxjvrku_mv_api_wjxjvrku%'"
2026/01/22 09:20:38 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_7045401232881674887 CURSOR FOR \n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('resync_failed_api_wjxjvrku' in flow_name) > 0\n\t\tAND error_message ILIKE '%while pushing to view e2e_test_api_wjxjvrku.resync_failed_api_wjxjvrku_mv_api_wjxjvrku%'" args=[]
2026/01/22 09:20:38 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('resync_failed_api_wjxjvrku' in flow_name) > 0\n\t\tAND error_message ILIKE '%while pushing to view e2e_test_api_wjxjvrku.resync_failed_api_wjxjvrku_mv_api_wjxjvrku%'" channelLen=0
2026/01/22 09:20:38 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7045401232881674887
2026/01/22 09:20:38 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7045401232881674887 records=1 bytes=8 channelLen=0
2026/01/22 09:20:38 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('resync_failed_api_wjxjvrku' in flow_name) > 0\n\t\tAND error_message ILIKE '%while pushing to view e2e_test_api_wjxjvrku.resync_failed_api_wjxjvrku_mv_api_wjxjvrku%'" rows=1 bytes=8 channelLen=0
2026/01/22 09:20:38 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7045401232881674887
2026/01/22 09:20:38 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7045401232881674887 records=0 bytes=0 channelLen=0
2026/01/22 09:20:38 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('resync_failed_api_wjxjvrku' in flow_name) > 0\n\t\tAND error_message ILIKE '%while pushing to view e2e_test_api_wjxjvrku.resync_failed_api_wjxjvrku_mv_api_wjxjvrku%'" rows=0 bytes=0 channelLen=0
2026/01/22 09:20:38 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:20:38 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="\n\t\tSELECT COUNT(*) FROM peerdb_stats.flow_errors\n\t\tWHERE error_type='error' AND position('resync_failed_api_wjxjvrku' in flow_name) > 0\n\t\tAND error_message ILIKE '%while pushing to view e2e_test_api_wjxjvrku.resync_failed_api_wjxjvrku_mv_api_wjxjvrku%'" rows=1 bytes=8 channelLen=0
    api_test.go:861: WaitFor wait for failed 2026-01-22 09:20:41.399539506 +0000 UTC m=+487.057419795
    api_test.go:879: WaitFor resync should have 2 rows 2026-01-22 09:20:51.94744796 +0000 UTC m=+497.605328249
    api_test.go:44: begin tearing down postgres schema api_wjxjvrku
2026/01/22 09:20:56 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_exxs0d7m.\"valid\" ORDER BY id"
2026/01/22 09:20:56 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_exxs0d7m.\"valid\" ORDER BY id"
2026/01/22 09:20:56 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_5604543073162539187 CURSOR FOR SELECT id,val FROM e2e_test_api_exxs0d7m.\"valid\" ORDER BY id" args=[]
2026/01/22 09:20:56 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_exxs0d7m.\"valid\" ORDER BY id" channelLen=0
2026/01/22 09:20:56 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5604543073162539187
2026/01/22 09:20:56 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5604543073162539187 records=2 bytes=19 channelLen=1
2026/01/22 09:20:56 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_exxs0d7m.\"valid\" ORDER BY id" rows=2 bytes=19 channelLen=0
2026/01/22 09:20:56 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5604543073162539187
2026/01/22 09:20:56 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_5604543073162539187 records=0 bytes=0 channelLen=0
2026/01/22 09:20:56 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_exxs0d7m.\"valid\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/01/22 09:20:56 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/22 09:20:56 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_exxs0d7m.\"valid\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:44: 
        	Error Trace:	.../flow/e2e/pg.go:149
        	            				.../flow/e2e/api_test.go:44
        	            				.../flow/e2eshared/e2eshared.go:38
        	            				.../hostedtoolcache/go/1.25.6.../src/testing/testing.go:1308
        	            				.../hostedtoolcache/go/1.25.6.../src/testing/testing.go:1572
        	            				.../hostedtoolcache/go/1.25.6.../src/testing/testing.go:1928
        	            				.../hostedtoolcache/go/1.25.6.../src/runtime/asm_amd64.s:1693
        	Error:      	failed to teardown postgres schema
        	Test:       	TestApiPg/TestResyncFailed
        	Messages:   	api_wjxjvrku: failed to drop replication slots: ERROR: replication slot "peerflow_slot_resync_failed_api_wjxjvrku" is active for PID 22216 (SQLSTATE 55006)
--- FAIL: TestApiPg/TestResyncFailed (137.92s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

jgao54 force-pushed the TIME64 branch 12 times, most recently from 3eb10eb to f334e8c Compare January 21, 2026 08:25

jgao54 changed the title ~~[test] revert-revert 'adding support of TIME64 datatype'~~ Adding support of TIME64 datatype (attempt 2) Jan 21, 2026

jgao54 force-pushed the TIME64 branch 2 times, most recently from ce8eae2 to 0d081bc Compare January 22, 2026 08:16

NehharShah and others added 3 commits January 21, 2026 22:16

adding support of TIME64 datatype

82329da

added backward compatibility test

8f54f37

fix time64 support

8bd22b8

jgao54 force-pushed the TIME64 branch from 0d081bc to 8bd22b8 Compare January 22, 2026 08:51

jgao54 changed the title ~~Adding support of TIME64 datatype (attempt 2)~~ Adding support of TIME64 datatype Jan 22, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Adding support of TIME64 datatype #3697

Adding support of TIME64 datatype #3697

Uh oh!

jgao54 commented Nov 15, 2025 •

edited

Loading

Uh oh!

codecov bot commented Nov 15, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Adding support of TIME64 datatype #3697

Are you sure you want to change the base?

Adding support of TIME64 datatype #3697

Uh oh!

Conversation

jgao54 commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 5 Tests Failed:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jgao54 commented Nov 15, 2025 •

edited

Loading

codecov bot commented Nov 15, 2025 •

edited

Loading